<html>
<head><meta charset="utf-8"><title>UI tests for stdlib · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html">UI tests for stdlib</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="208820220"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208820220" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208820220">(Sep 02 2020 at 13:07)</a>:</h4>
<p>I've noticed that we recently added a bunch of <code>run-pass</code> tests for stdlib functionality, like this one: <a href="https://github.com/rust-lang/rust/pull/76206/files#diff-48ce30297662d3fd30868a0d48aee6c3">https://github.com/rust-lang/rust/pull/76206/files#diff-48ce30297662d3fd30868a0d48aee6c3</a></p>
<p>is this a right thing to do? My gut reaction is no -- I feel that run-pass tests are very costly (as they require linking a separate binary, and launching a separate process). </p>
<p>Right now test suite is pretty fast (or rather, everything else about build is very slow), but I worry that we might end up with a slow and unoptimizable test-suite.</p>
<p>I would suggest moving all such tests to <code>#[test]</code> tests, which are in principle much faster.</p>



<a name="208820296"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208820296" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208820296">(Sep 02 2020 at 13:08)</a>:</h4>
<p>Am I missing something that makes <code>run-pass</code> test more preferable/less costly here?</p>



<a name="208821013"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208821013" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208821013">(Sep 02 2020 at 13:13)</a>:</h4>
<p>run-pass tests are sometimes nice because they give you easy expect! testing if you need to compare outputs</p>



<a name="208821128"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208821128" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208821128">(Sep 02 2020 at 13:14)</a>:</h4>
<p>I don't know why that test is run-pass though, it seems like it should indeed just be a (series of) unit tests</p>



<a name="208822066"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208822066" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208822066">(Sep 02 2020 at 13:20)</a>:</h4>
<p>Yeah, run-pass tests have some advantages for stdlib, but <code>#[test]</code> tests even more so (a big one is that <code>./x.py test library/std</code> works as expected). </p>
<p>Basically, I think a useful guideline is "std/core tests go to ./library, unless you are doing something <strong>really</strong> weird". Should we document this issue somewhere? </p>
<p>Also cc <span class="user-mention" data-user-id="118594">@ecstatic-morse</span> as the reviewer of the said PRs.</p>



<a name="208823502"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208823502" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208823502">(Sep 02 2020 at 13:30)</a>:</h4>
<p>I was thinking that perhaps we could have a highfive warning if you modify src/test without changing compiler/ or something like that</p>



<a name="208823546"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208823546" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208823546">(Sep 02 2020 at 13:31)</a>:</h4>
<p>obviously not a hard error -- you could be adding tests without compiler changes -- but seems relatively likely to warn on that</p>



<a name="208824042"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208824042" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208824042">(Sep 02 2020 at 13:34)</a>:</h4>
<p>(please also include <code>src/librustdoc</code> in 'directories that need modifying <code>src/test</code>')</p>



<a name="208828058"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208828058" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208828058">(Sep 02 2020 at 14:02)</a>:</h4>
<p>I am not sure -- adding regression tests is pretty common, so I would thing that FPR would be higher than TPR.</p>



<a name="208828102"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208828102" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208828102">(Sep 02 2020 at 14:02)</a>:</h4>
<p>Not sure what are definitely impactful action items here, except that look through each ui test if it makes sense :)</p>



<a name="208862790"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208862790" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dylan MacKenzie (ecstatic-morse) <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208862790">(Sep 02 2020 at 17:56)</a>:</h4>
<p>I have no objection to making these kinds of tests into unit tests and can recommend that people do this in the future.  However, there are a large number of them already in <code>src/test/ui</code> (many were moved from the old <code>run-pass</code> directory), so enforcing this retroactively will require a decent chunk of time for a pretty small benefit.</p>



<a name="208865587"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208865587" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dylan MacKenzie (ecstatic-morse) <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208865587">(Sep 02 2020 at 18:18)</a>:</h4>
<p>It might be good to have an open issue for this?</p>



<a name="208887695"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208887695" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208887695">(Sep 02 2020 at 21:05)</a>:</h4>
<p>FWIW I've found that <code>E-easy</code> <code>E-mentor</code> issues get a <em>lot</em> of people from the community helping on them :)</p>



<a name="208887721"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208887721" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208887721">(Sep 02 2020 at 21:05)</a>:</h4>
<p>so if there was a tracking issue with a list I imagine it would get done sooner than expected</p>



<a name="208903919"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208903919" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208903919">(Sep 02 2020 at 23:54)</a>:</h4>
<blockquote>
<p>for a pretty small benefit.</p>
</blockquote>
<p>I kind of feel that the benefit would be substantional over the lifetime of the project, but I also don't feel motivated enough right now to spearhead this effort :)</p>
<p>I'd be happy  to do reviews for moved tests, if someone else creates the issue :-)</p>



<a name="208907821"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208907821" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208907821">(Sep 03 2020 at 00:48)</a>:</h4>
<p>I'll make an issue</p>



<a name="208908078"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208908078" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208908078">(Sep 03 2020 at 00:52)</a>:</h4>
<p><span class="user-mention" data-user-id="133169">@matklad</span> do you have an example PR I could reference in the issue?</p>



<a name="208908433"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208908433" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208908433">(Sep 03 2020 at 00:58)</a>:</h4>
<p>and do you have recommendations for how to find tests that could be made into unit tests?</p>



<a name="208908448"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208908448" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208908448">(Sep 03 2020 at 00:59)</a>:</h4>
<p>hmm I guess as a very rough first approximation they'd need <code>run-pass</code></p>



<a name="208908725"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208908725" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208908725">(Sep 03 2020 at 01:03)</a>:</h4>
<p>and they couldn't have <code>WARN</code> either because those can't be tested with unit tests</p>



<a name="208909166"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208909166" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208909166">(Sep 03 2020 at 01:11)</a>:</h4>
<p><a href="https://github.com/rust-lang/rust/issues/76268">https://github.com/rust-lang/rust/issues/76268</a></p>



<a name="208909207"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208909207" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208909207">(Sep 03 2020 at 01:12)</a>:</h4>
<p>feel free to edit the description, not sure how clear it is</p>



<a name="208909747"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208909747" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208909747">(Sep 03 2020 at 01:22)</a>:</h4>
<p>opened <a href="https://github.com/rust-lang/highfive/issues/286">https://github.com/rust-lang/highfive/issues/286</a> too</p>



<a name="208920041"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208920041" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> CraftSpider <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208920041">(Sep 03 2020 at 04:36)</a>:</h4>
<p>I have interest in helping out with this, to dip my toes into the contribution process. Would this be the right place to ask questions about where to put things/how to convert some of these?</p>



<a name="208920518"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208920518" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208920518">(Sep 03 2020 at 04:47)</a>:</h4>
<p>Sure, this is a good spot :)</p>



<a name="208921079"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208921079" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> CraftSpider <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208921079">(Sep 03 2020 at 04:58)</a>:</h4>
<p>So, because I want to make sure I've got the basics: it looks like <code>ui/array-slice-vec/vec-slice.rs</code> is just checking that one can take a slice of a vec, and that it works. Would that be a good candidate to move into <code>alloc/tests/vec.rs</code> as, say, <code>test_slice_from_ref</code> (To match <code>test_slice_from_mut</code>)? As there appears to not be that test in there already</p>



<a name="208947897"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208947897" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208947897">(Sep 03 2020 at 10:38)</a>:</h4>
<p><span class="user-mention" data-user-id="319144">@Rune Tynan</span> yeah, I think that that test should be fine to move, although it's not entirely clear cut. </p>
<p>One way to decide here is to look at the commit/PR which added the test. If the commit did some changes to the stdlib, and then added a UI test, than the test almost certainly should be moved to stdlib. </p>
<p>if the commit changed <em>compiler</em>, than that means the the tests actually tests something about the compiler, and probably should not be moved.</p>
<p>In this particular case, this test was there from time immemorial, so it's unclear if it intends to check stdlib or compiler just from the git history. </p>
<p>But today, it's pretty clear that it just exercises vec's and slice's API, so it should live in <code>/library</code> unit tests</p>



<a name="208950444"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/UI%20tests%20for%20stdlib/near/208950444" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/UI.20tests.20for.20stdlib.html#208950444">(Sep 03 2020 at 11:06)</a>:</h4>
<p>(Added this info to the issue description )</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>